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AFTER FINAL ACTION 
PLEASE EXPEDITE 



IN THE UNITED STATES PATENT AND TRADEMARK OFFICE 
BEFORE THE BOARD OF PATENT APPEALS AND INTERFERENCES 



In re Application of Krishnamachari, S. Atty. Docket No.: PHA 23-431A 

Serial No.: 09/934,962 Group Art Unit: 2623 

Filed: 8/22/2001 Examiner: Wu, J. 

Title: COLOR QUANTIZATION AND SIMILARITY MEASURE FOR CONTENT 

BASED IMAGE RETRIEVAL 

REPLY BRIEF 

Honorable Commissioner for Patents 
Alexandria, VA 22313-1450 

Sir: 

This is a reply to the Examiner's Answer to the Applicant's Appeal Brief in the 
subject application, dated 8 April 2003. 



GROUPING OF CLAIMS 

The Examiner's Answer states that all claims stand or fall together. The Applicant 
respectfully notes that in the Applicant's Brief, the Applicant stated that claims 1-5 and 
11-15 stand or fall together, and that claims 6-10 and 16-20 stand or fall together. As 
noted in the Applicant's Brief, claims 1-5 and 1 1-15 are patentably distinct from claims 6- 
10 and 16-20 because claims 1-5 and 1 1-15 claim a method and system for characterizing 
images, whereas claims 6-10 and 16-20 claim a method and system for comparing two or 
more images. 

CLAIMS APPEALED 

The Examiner's Answer notes that the appealed claims in the Applicant's Brief 
were not correct, because a requested amendment under 37 CFR 1.116 was not admitted. 
Attached is a corrected set of claims. 
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GROUNDS OF REJECTION 

The Examiner disagrees with the Applicant's assertion that "Sato does not perform 
a comparison of frequency of occurrences of colors in each region of an image, as 
specifically claimed 11 (Examiner's answer, page 3, lines 14-15). 

The Applicant respectfully notes that in Sato, a comparison of images uses the 
index table of FIG. 45. The Applicant agrees with the Examiner that the index table of 
FIG. 45 is created based on a histogram of colors in each region, as illustrated in FIG. 43 
and the flowchart of FIG. 44, and agrees with the Examiner that a histogram represents 
frequency of occurrences. The Applicant also agrees with the Examiner that Sato 
searches for images in the database by a comparison of Sato's index table (Examiner 1 
Answer, page 4, lines 5-6). However, the Applicant respectfully maintains that the index 
table of FIG. 45 does not contain frequencies of occurrences, and thus the use of the 
index table to compare images does not include a comparison of frequencies of 
occurrences of colors. 

The columns of Sato's index table are different levels of "resolution"; the larger 
the block size, the lesser the resolution. The rows of Sato's index table are each of the 
different colors contained in the image. The entries within the table are identifiers of 
regions of the image that exhibit at least a threshold amount of a particular color at the 
given resolution. As specifically illustrated at S175 of Sato's FIG. 44, the entry at column 
Ci and resolution R in the index table is "IMAGE id, REGION wf ', where id is an 
abbreviation of identifier, and not frequency of occurrence of color Ci. If the histogram 
of FIG. 43 shows that a particular color in a region exceeds a resolution-dependent 
threshold value, the identifier of that region is entered in the index table of FIG. 45 for 
that color at that resolution. 

In Sato's example entry in FIG. 45, "IMAGE 1, REGION 1" contains at least a 
threshold amount of color CI, and "IMAGE 2, REGION 3" also contains at least a 
threshold amount of color CI . It is impossible to determine from Sato's index table 
whether "IMAGE 1, REGION 1" has fewer or more pixels of color 1 than "IMAGE 2, 
REGION 3". The Examiner asserts that the index to this table, color CI, contains both the 
color and its frequency of occurrence (Examiner's Answer, page 4, lines 7-8). The 
Applicant must then ask, if this assertion is correct, does the color CI represent the 
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frequency of occurrence of the color in "IMAGE 1, REGION 1" or in "IMAGE 2, 
REGION 3"? Obviously, except when the frequency of occurrences of each are identical, 
it cannot represent the frequency of occurrence of both, because if it represented the 
frequency of occurrence of colors in "IMAGE 1, REGION 1", then the entry of "IMAGE 
2, REGION 3" in this row is incorrect; likewise, if it represents the frequency of 
occurrence in "IMAGE 2, REGION 3", then the entry of "IMAGE 1, REGION 1" is 
incorrect. 

Consider, for example, a threshold value of 5 being defined for a block size ol*16. 
If a region of 16 pixels includes five or more pixels of a given color, the identifier of that 
region is entered in the "BLOCK SIZE: 16" column at the row corresponding to the given 
color. Of particular note, in this example, if "IMAGE 1, REGION 1" is a region that 
contains five pixels of color CI, the identifier of this region will appear in the index table, 
if "IMAGE 2, REGION 3" contains all sixteen pixels of color CI, it will also appear in 
the index table. The entries in column CI merely conveys that both regions have a 
threshold amount of color CI, it does not convey the actual frequency of occurrence, and 
thus the use of this table for subsequent comparisons, as taught by Sato, is not a 
comparison of the determined frequencies of occurrences, as specifically claimed in the 
Applicant's claims 6-10 and 16-20. 

In Sato, when images are compared, the comparison is based on whether the 
corresponding regions contain the same colors, as identified in the index table, and not on 
whether they contain the same amount of each color, as taught and claimed by the 
Applicant. 

In the Applicant's invention, a 16-pixel region of a first image that contained live 
pixels of a color would be distinguished from a 16-pixel region of a second image that 
contained sixteen pixels of the color, because the frequency of occurrences of each color 
is compared, and the difference in the frequencies of occurrences forms the measure of 
similarity between the images. As illustrated in the Applicant's FIG. 4, block 434, and 
discussed in the specification at page 9, line 30 through page 10, line 3, in the example 
embodiment, the SumP and DiffP terms are specifically a difference between the 
frequency of occurrences, or proportion, of colors in each partition. 
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The Applicant respectfully maintains that comparing frequencies of occurrences 
of colors in each region, as specifically claimed in the Applicant's claims 6-10 and 16-20, 
is not taught by Sato. When Sato compares images, the comparison uses the index table, 
which, although it was created using the frequencies of occurrences of each color, docs 
not contain frequencies of occurrences, and thus Sato's comparison cannot be said to be a 
comparison of frequencies of occurrences. 

Further, in the Examiner's Answer, the Examiner asserts that Sato teaches a 
comparison of the frequencies of occurrences of colors at column 24, lines 4-26. The 
Applicant respectfully notes that at column 24, lines 4-26, Sato teaches comparing the 
similarity of colors, and not the frequency of colors. Throughout these cited lines, Salo 
specifically refers to the calculation of color distances; nowhere in these cited lines does 
Sato refer to the proportions of colors or their frequencies of occurrences. 

In claims 1-5 and 1 1-15, the Applicant specifically claims a characterization of 
the image that includes "a plurality of measures that are proportional to the frequency of 
occurrence of a plurality of colors". The Applicant respectfully notes that although Sato's 
histogram of FIG. 43 provides a plurality of measures that are proportional to the 
frequency of occurrence of a plurality of colors, this histogram is not included in the 
characterization of the image. Sato's characterization of the image is the index table of 
FIG. 45, and not the histogram of FIG. 43. Although Sato's entries in the index table are 
based on the histogram of FIG. 43, they are not proportional to the frequency of 
occurrence of a plurality of colors, as specifically claimed by the Applicant, as the term 
proportional is generally used in the art, consistent with the use of the term in the 
Applicant's specification. 



The Applicant respectfully maintains that Sato uses the index table to 
characterize an image, and respectfully maintains that Sato's index table does not contain 
frequency of occurrences of colors. Because Sato specifically teaches the determination 
of frequencies of occurrences of colors, and then creates a characterization based on the 
frequencies of occurrences but not containing the frequencies of occurrences, or any 
measures that are proportional to the frequencies of occurrences, the Applicant 
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respectfully maintains that Sato does not teach a characterization that includes measures 
proportional to frequencies of occurrences, as specifically claimed in claims 1-5 and 1 1- 
1 5, and that Sato does not teach a comparison of images based on a comparison of 
frequencies of occurrences, as specifically claimed in claims 6-10 and 16-20. 



CONCLUSIONS 

Because Sato stores only an identifier of each region that contains a threshold 
amount of each color in the index table that characterizes the image, and does not store a 
measure proportionate to the frequency of occurrences of each color in each region in this 
index table, the Applicant respectfully requests that the Examiner's rejection of claims 1- 
5 and 11-15 under 35 U.S.C. 102(e) be reversed by the Board, and the claims be allowed 
to pass to issue. 

Because Sato stores only an identifier of each region that contains a threshold 
amount of each color in the index table, and does not store the frequency of occurrence of 
each color in each region for subsequent comparisons using the index table, the Applicant 
respectfully requests that the Examiner's rejection of claims 6-10 and 16-20 under 35 
U.S.C. 102(e) be reversed by the Board, and the claims be allowed to pass to issue. 



Respectfully submitted, 

Robert M. McDermott, Attorney 
Registration Number 41,508 
804-493-0707 



CERTIFICATE OF MAILING OR TRANSMISSION 
It is hereby certified that, on the date shown below, this correspondence is being: 

| ( deposited with the United States Postal Service with sufficient postage as first-class mat! in an envelope addressed 
to: Mail Stop: Appeal, Commissioner for Patent*, P.O. Box 1450, Alexandria, VA 22313-1450. 

|XJ transmitted by facsimile to the United States Patent and Trademark Office at 703-872-9314. 



On 8 June 2003 



By, 
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APPENDIX 
CLAIMS ON APPEAL 

1 . A method for characterizing an image comprising: 

partitioning the image into a plurality of partitions, each partition including a 
plurality of pixels, each pixel having a color, 

determining a frequency of occurrence of each color of the plurality of pixels 
within each partition, and 

creating a characterization that includes a plurality of measures that are 
proportional to the frequency of occurrence of a plurality of colors. 

2. The method of claim 1, further including 

quantizing an encoded color of each pixel lo provide the color of each pixel. 

_ 3. The method of claim 2, further including 

identifying a plurality of populous colors, based on the frequency of occurrence of 
each color, and 

the plurality of measures includes proportions of each of the plurality of populous 
colors in each partition. 

4. The method of claim 2, wherein 

quantizing the encoded color includes 

identifying a set of color centers, and 

determining the color of each pixel based upon a color distance between 
the encoded color of each pixel and each of the set of color centers. 

- 5. The method of claim 1 , further including 

identifying a plurality of populous colors, based on the frequency of occurrence of 
each color, and 

the plurality of measures includes proportions of each of the plurality of populous 
colors in each partition. 
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6. A method of comparing a first image to a second image, comprising 

partitioning the first image into a plurality of first partitions, each first partition 
including a plurality of first pixels, each first pixel having a color, 

determining a frequency of occurrence of each color of the plurality of first pixels 
within each first partition, 

partitioning the second image into a plurality of second partitions, each second 
partition including a plurality of second pixels, each second pixel having a color, 

determining a frequency of occurrence of each color of the plurality of second 
pixels within each second partition 

comparing the frequency of occurrence of a select set of colors in each first 
partition with the frequency of occurrence of a corresponding select set of colors in each 
second partition. 

7. The method of claim 6, further including 

quantizing an encoded color of each pixel of the plurality of first pixels to provide 
the color of each pixel of the plurality of first pixels. 

8. The method of claim 7, further including 

identifying a plurality of first populous colors, based on the frequency of 
occurrence of each color of the plurality of first pixels, and 

identifying a plurality of second populous colors, based on the frequency of 
occurrence of each color of the plurality of second pixels; and 

wherein 

the select set of colors in each first partition corresponds to the plurality of first 
populous colors, and 

the corresponding set of colors in each second partition is based upon a color 
difference between each of the plurality of second populous colors and the plurality of 
first populous colors. 
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9. The method of claim 7, wherein 

quantizing the encoded color includes 

identifying a set of color centers, and 

determining the color of each pixel based upon a color distance between 
the encoded color of each pixel and each of the set of color centers. 

10. The method of claim 6, further including 

identifying a plurality of first populous colors, based on the frequency of 
occurrence of each color of the plurality of first pixels, and 

identifying a plurality of second populous colors, based on the frequency of 
occurrence of each color of the plurality of second pixels; and 

wherein 

the select set of colors in each first partition corresponds to the plurality of first 
populous colors, and 

the corresponding set of colors in each second partition is based upon a color 
difference between each of the plurality of second populous colors and the plurality of 
first populous colors. 
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1 1. A system for characterizing an image comprising; 

a partitioner that is configured to partition the image into a plurality of partitions, 
each partition including a plurality of pixels, each pixel having a color, and 

an accumulator that is configured to determine a frequency of occurrence of each 
color of the plurality of pixels within each partition, and 

wherein 

the system is configured to create a characterization of the image that includes a 
plurality of measures that are proportional to the frequency of occurrences of a plurality 
of colors. 

12. The system of claim 1 1, further including 

a quantizer that is configured to quantize an encoded color of each pixel to 
provide the color of each pixel. 

- 13. The system of claim 12, wherein 

the plurality of measures are based on the frequency of occurrence of each of a 
plurality of populous colors in each partition. 

14. The system of claim 12, wherein 

the quantizer is configured to quantize the encoded color based upon a color 
distance between the encoded color of each pixel and each of a set of color centers. 

15. The system of claim 11, wherein 

the plurality of measures are based on the frequency of occurrence of each of a 
plurality of populous colors in each partition. 
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1 6. A system for comparing a first image to a second image, the system comprising: 

a similar color determinator that is configured to determine 

a mapping between a first set of colors of pixels of the first image and a 
second set of colors of pixels of the second image, based on a color distance between 
each of the first set of colors and each of the second set of colors, 

the mapping thereby providing a corresponding color in the second set of 
colors for each color in the first set of colors, and 

a similarity determinator that is configured to determine an image similarity 
measure based on a comparison of a frequency of occurrence of pixels of each of the first 
set of colors and a frequency of occurrence of pixels of each of the corresponding colors 
in the second set of colors. 

17. The system of claim 16, wherein 

the first image is partitioned into a plurality of first partitions, 

the second image is partitioned into a plurality of second partitions, 

the similar color determinator is configured to determine the mapping between the 

first and second sets of colors of pixels for each partition of the plurality of first and 

second partitions, and 

the similarity determinator is configured to determine a plurality of similarity 

measures based on the comparison of the frequencies of occurrence of pixels of each of 

the first and second set of colors for each partition of the plurality of first and second 

partitions, and further includes 

an accumulator that is configured to provide the image similarity measure 

based on a composite of the plurality of similarity measures corresponding to each 

partition of the first and second partitions. 

18. The system of claim 17, wherein 

the similarity determinator is further configured to determine the similarity 
measure based upon the color distances between each of the first set of colors and the 
corresponding color in the second set of colors. 
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19. The system of claim 16, wherein 

the first set of colors of the pixels of the first image is based on a quantization of 
encoded colors of the pixels of the first image. 

20. The system of claim 17, wherein 

the quantization of encoded colors is based on a color distance between the 
encoded color of each pixel and each of a set of color centers. 
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